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Abstract 

Background: Patient-reported outcome measurement (PROM) plays an increasingly important role in palliative 
care. A variety of measures exists and is used in clinical care, audit and research. However, little is known about 
professionals' views using these measures. The aim of this study is to describe the use and experiences of palliative 
care professionals with outcome measures. 

Methods: A web-based online survey was conducted in Europe and Africa. Professionals working in clinical care, 
audit and research in palliative care were invited to the survey via national palliative care associations and various 
databases. Invitation e-mails were sent with a link to the questionnaire. 

Results: Overall participation rate 42% (663/1592), overall completion rate 59% (392/663). The majority of 
respondents were female (63.4%), mean age 46 years (SD 9). 68.1% respondents from Europe and 73.3% from 
Africa had experiences with outcome measures in palliative care. Non-users reported time constraints, burden, lack 
of training and guidance as main reasons. In clinical care/audit, assessment of patients' situation, monitoring 
changes and evaluation of services were main reasons for use. Choice of OMs for research was influenced by 
validity of the instrument in palliative care and comparability with international literature. Main problems were 
related to patient characteristics, staff, and outcome measures. Participants expressed the need for more guidance 
and training in the use of PROMs. 

Conclusions: Professionals need more support for the use and implementation of PROMs in clinical practice and 
research through training and guidance in order to improve patient care. 




HEALTH AND QUALITY 
OF LIFE OUTCOMES 



Background 

Patient-reported outcome measurement plays an 
increasingly important role in health care in allowing 
patients to assess the effect and quality of their care [1]. 
Patient reported outcome measures (PROMs) are used 
in clinical care (e.g. assessing the health status and 
needs of patients in a hospital at admission), audit (qual- 
ity assurance of services) and research (e.g. studying the 
effectiveness of an intervention). The measurement of 
effects and outcomes on patients is also central to end- 
of-life (eol) care and the conduct of research in eol care. 



* Correspondence: Claudia.bausewein@kcl.ac.uk 

1 King's College London, Cicely Saunders Institute, Department of Palliative 

Care, Policy and Rehabilitation, London, UK 

Full list of author information is available at the end of the article 

(3 BioMed Central 



Palliative care services should be committed to excel- 
lence and high quality of care requiring the regular and 
systematic evaluation of the processes of care and mea- 
surement of outcomes data using validated instruments 
[2]. In the future, commissioning of services will be 
based on outcomes rather than activity and PROMS will 
have a central role in this [3]. 

Although initiatives were started over the last decade 
to improve outcome measurement in palliative care 
[2,4,5] a clear roadmap or uniform approach to measur- 
ing outcomes in eol clinical care and research is lacking 
[2]. In consequence, there are differences in the inter- 
pretation of results of studies, meta-analyses are often 
limited because no core measure is used, and, impor- 
tantly, some studies fail because they have used 
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inappropriate measures without adequate responsiveness 
to change. In clinical care, formalised assessments of 
patient-reported outcomes may increase clinicians' 
attention to patient concerns which are often overlooked 
[6]. Recent reports from a clinical workshop indicate 
that assessment and measurement are important 
domains, and form a priority area in eol research [7]. 
Thus, within the midst of an increasing ageing European 
population, and a continued underspend (0.5%) in eol 
cancer research [8], a common approach regarding out- 
come measurement is urgently required. 

Professionals play a central role in the utilisation of 
PROMs, but besides a few studies on professionals' 
knowledge of quality of life instruments [9-11] little is 
known about their experiences in the selection and 
implementation of PROMs in clinical care and research. 
However, their views can give important insights how 
outcome measurement could be promoted and 
harmonised. 

PRISMA ("Reflecting the Positive diveRsities of Eur- 
opean priorities for reSearch and Measurement in end- 
of-life cAre") is a three-year project funded by the Eur- 
opean Commission with a focus on co-ordinating out- 
come measurement in palliative care [12]. One of the 
aims is to map and harmonise approaches and experi- 
ences in eol care measurement. Within PRISMA, one 
programme of activities concentrates on the experiences 
of professionals using PROMs in eol care, and on two 
widely used outcome measures in particular, the Pallia- 
tive Care Outcome Scale (POS) and the Support Team 
Assessment Schedule (STAS). The STAS is a tool to 
evaluate the work of palliative care support teams mea- 
suring patient symptoms, anxiety and insight, family 
anxiety and insight, quality of communication with 
health care professionals and carers, and need for practi- 
cal support [13]. The POS consists of ten items asses- 
sing physical symptoms, emotional, psychological and 
spiritual needs, and provision of information and sup- 
port both from a patient's and professional carer's per- 
spective [14]. Although PRISMA is a European project, 
one of the members is the African Palliative Care Asso- 
ciation (APCA), as research and outcome measurement 
play an important role in the young history of African 
palliative care [15]. 

We report here a study aimed to describe the use and 
experiences with PROMs of professionals working in 
palliative care in Europe and Africa. In particular we 
aimed to describe: 

1. reasons of those not using PROMs in palliative 
care; 

2. practice of use (purposes, frequencies etc.) of 
PROMs in clinical care/audit and research; 



3. views of users regarding advantages and chal- 
lenges of using PROMs; 

4. participants' views on further development of 
PROMs. 

Our study responds to the call for harmonising 
research and best practice across Europe, and in doing 
so addresses an urgent need for improved measurement 
in eol care. 

Methods 

A web-based online survey was conducted following the 
Checklist for Reporting Results of Internet E-Survey 
(CHERRIES) [16]. Survey development included expert 
review within the PRISMA group, and the electronic 
version was piloted in seven European countries (Aus- 
tria, Germany, Italy, Netherlands, Norway, Portugal and 
UK) with 20 professionals working in palliative care. 
The questionnaire included questions on: 

♦ use of tools in general including questions for 
those not using tools; 

♦ use of tools and PROMs in clinical care/audit and 
research (purpose, selection, frequency of use, pro- 
blems and advantages using tools, overall 
experience); 

♦ further development of tools and resources (char- 
acteristics of ideal tool, content of web-based 
resources etc.); 

♦ respondent information (demographics, profession, 
setting, country etc.). 

Additional open questions gave respondents the 
opportunity to share their views and experiences. 

The survey was conducted in English due to resource 
constraints. Open questions gave respondents the 
opportunity to share their views and experiences in 
their native language. Following a competitive tendering 
process as per European Commission guidelines, the 
Centre of Evaluation and Methods (ZEM) at the Univer- 
sity of Bonn/Germany http://www.zem.uni-bonn.de was 
commissioned to conduct the online survey. Special 
attention was given to a user-friendly outlook, support 
features during completion of the survey such as alerts 
for unanswered questions, "back buttons" for reviewing 
answers and data protection to ensure confidentiality 
and anonymity. "Adaptive questioning" was used where 
certain questions were only displayed based on the 
responses to other items to reduce the number and 
complexity of the questions. To improve completion 
rate, only one question was displayed per page on the 
screen. Overall, the questionnaire contained 59 ques- 
tions (screens). 
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A Secure Sockets Layer Virtual Private Network (SSL 
VPN server) was used to ensure safety and anonymity of 
participants. Contact details of participants and answers 
to the survey were saved on two different servers. It was 
decided not to include a link with a password. Providing 
passwords would have enabled participants an individua- 
lised access to the survey in order to come back to the 
questionnaire. However, data protection did not allow 
use of member lists of the national palliative care asso- 
ciations. In these scenarios, the national associations 
sent the invitation e-mails to around 2000 people. 

Sampling 

The sampling frame for the survey were professionals 
working in palliative care (e.g. doctors, nurses, other 
professionals) either in clinical care, audit or research in 
Europe or Africa. 

The following institutions and databases were used to 
sample potential participants in order to reach a maxi- 
mum of experiences and diversity of professional back- 
grounds (see Table 1). 

7. National Palliative Care Associations in Europe 

Palliative care associations in eight countries of the 
PRISMA collaboration (Belgium, Germany, Italy, Neth- 
erlands, Norway, Portugal, Spain and UK) were invited 



to participate in the survey. To adjust for the different 
size of countries and associations, a weighted random 
sample of 20% was drawn from each group if the total 
number of clinicians (physicians, nurses and other) in 
the associations was over 500 members and 33% if the 
total number was less than 500 members. 

2. African Palliative Care Association (APCA) 

The African Palliative Care Association (APCA) is a 
membership organisation for individuals and organisa- 
tions interested in and working in palliative care in 
Africa. APCA has 475 individual contacts from 25 Afri- 
can countries. As the list does not contain information 
on professional background of members, invitation e- 
mails were sent to all members but those who were not 
clinicians were asked to feed this information back to 
the APCA office. This procedure should help to assess 
the number of non-respondents, taking into account 
those who are non-clinicians. 

3. Other international contacts 

The Department of Palliative Care, Policy & Rehabilita- 
tion, King's College London's database containing regis- 
tered POS users was used for the invitations as well as a 
list of POS and STAS authors who were identified from 
a systematic review on the use of POS and STAS [17]. 
Within PRISMA, a list of 61 chairs and researchers in 



Table 1 European Palliative Care Associations participating in the online survey 



Country 


Association 




No. of members 


Sample 


Belgium 


FPCF 


Federatie Palliatieve Zorg Vlaanderen 


About 200 (mixed individuals & 
organisations) 


33% 




FWSP 


Federation Wallonne des Soins Palliatifs 






Germany 


DGP 


Deutsche Gesellschaft fur Palliativmedizin e.V. 


3,100 


20% 


Italy 


SICP 


Societa Italiana di Cure Palliative 


2,454 (1092 doc, 829 nurse, 375 other 
= 2,296) 


20% 


Nether-land 


NPTN 


Netherlands Palliative Care Network for 
Terminally III Patients 


102 (mixed including organisations) 


33% 


Norway 


NFPM 


Norwegian Association for Palliative Medicine 


112 (only physicians) 


33% 


Portugal 


APCP 


Portuguese Association of Palliative Care 


406 


33% 


Spain 


SCBCP 


Soc. Catalano-Balear de Cures Palliatives 


360 


33% 




SECPAL 


Sociedad Espahola de Cuidados Paliativos 


1500 


20% 


UK 


APM 


Association for Palliative Medicine of Great Britain 
& Ireland 


1,030 (only physicians) 


20% 




APCSW 


Association of Palliative Care Social Workers 


309 (only social workers) 


33% 




RCN 


Royal College of Nursing - Palliative Nursing 
Group 


3,196 (only nurses) 


No sampling 
possible* 






UK Nurse consultants 


28 


100% 


Other 
contacts 


POS database 


Department of Palliative Care, Policy & 
Rehabilitation, KCL 


211 


100% 




POS & STAS authors 


Systematic review of literature 


61 


100% 




Pall care academics in 
Europe 


Chairs & other researchers in palliative medicine 
in Europe 


61 


100% 




PRISMA members 




38 


100% 



# invitation letter sent to all and request for link to survey demanded 
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palliative medicine in Europe was established [18]. All 
contacts of this list were invited to participate in the 
survey. Finally, all 38 members of the PRISMA consor- 
tium were invited to participate in the online survey. 

Data collection 

The European online survey was conducted in October 
and November 2009 with two reminder emails. Invita- 
tion e-mails with information about the content and the 
purpose of the survey, the time for completion, the prin- 
cipal investigator and a link to the web-based survey 
were sent out to potential participants either by the pal- 
liative care associations or directly to named individuals 
from the POS database, the POS and STAS authors, and 
European palliative care academics (text of invitation e- 
mail in Additional file 1). Overall, 2000 e-mails were 
sent to professionals working in palliative care in Eur- 
ope. On the first page of the online questionnaire, infor- 
mation about the survey and instructions how to fill in 
the questionnaire were provided (see Additional file 2). 

Due to some delays with ethics approval for the Afri- 
can part of the survey, the African online survey was 
conducted in January/ February 2010 with one reminder. 
Overall, 487 invitation e-mails were sent out to APCA 
members initially. Sixty-five emails were returned to the 
APCA office. For the reminder, 422 e-mails were sent. 

To increase participation in the survey we offered a 
low-cost prize draw to respondents with book tokens 
for a random number of respondents. 

Analysis 

Following the CHERRIES checklist, the participation 
rate was calculated dividing the number of replies to the 
first question by the number of unique site visitors 
(defined as those visiting the first page of the online sur- 
vey) and the completion rate was calculated by the 
number of participants answering the first question 
divided by the number of people submitting the last 
question [16]. Descriptive analysis of all questions was 
conducted using frequencies for categorical data and 
means and SD for continuous variables. Answers from 
open-ended questions were analysed using content ana- 
lysis [19]. First, the answer options were collated and 
then coded looking for emerging themes. We analyzed 
and report responses for Europe and Africa separately, 
not to compare responses for these continents, but to 
give a picture in both regions of the state of science in 
PROMs and future needs. As the POS database and 
POS author list included people from all over the world, 
17 respondents from the first survey came from outside 
Europe (Malaysia, Taiwan each n = 1; Australia, Brazil, 
US, each n = 2; Canada, Japan, Thailand each n = 3). 
We grouped these participants in two groups (Australia/ 
Canada/Japan/US (ACJU), n = 10; Brazil/Malaysia/ 



Taiwan/Thailand (BMTT), n = 7) in relation to their 
health care system and socio-economic status. As the 
data from Europe and ACJU and also the data from 
Africa and BMTT were similar regarding the demo- 
graphic variables we merged these groups accordingly. 
SPSS Statistics version 17.0 was used for quantitative 
and NViVo 8 to assist in qualitative data analysis. 

Ethics 

Ethic approval was obtained from the Research Ethics 
Committee at King's College London (BDM/08/09-102). 
We were advised that further ethics approval from all 
other countries was not needed as this survey did not 
include patients but only professionals. For Africa, ethics 
approval was obtained from the Ethics Committee of the 
Ugandan National Council for Science and Technology 
(IS 62). 

Results 

1592 unique site visitors (1291 in Europe and 301 in 
Africa) were counted on the first page of the survey 
(which gave more specific information about the sur- 
vey). The overall participation rate was 42% with 663/ 
1592 professionals answering the first question and 392/ 
663 completing the last question (completion rate 59%). 
In Europe the participation rate was 38% (495/1291) 
and the completion rate 63% (311/495), whereas in 
Africa the participation rate was 56% (168/301) and the 
completion rate 48% (81/168). 

Sample characteristics 

Demographic data was available from 379/663 partici- 
pants. 63.4% of respondents were female; mean age of 
all respondents was 46 years (SD 9). Both in Europe and 
in Africa, the majority had a clinical background, about 
10% were researchers and about 19% of respondents in 
Europe and 26% in Africa had both a clinical and a 
research background (see Table 2). In both continents, 
about two thirds of participants had more than five 
years experience in palliative care. 

In Europe, physicians constituted the largest group 
both of clinicians and researchers. Every fourth clinical 
respondent and every eighth researcher had a nursing 
background. In Africa, nurses were the largest group of 
clinicians but medicine, nursing and social science were 
the most common backgrounds of researchers. About 
30% of researchers in Europe and over 50% in Africa 
had a background other than medicine or nursing, such 
as psychology, social science, or the health sciences. 78% 
of respondents in Europe and 64% in Africa came from 
a specialist palliative care background. 

Participants came from a variety of European coun- 
tries with 37% from the UK, 15% from Italy, 12% from 
Germany, and 7% each from Spain and Portugal. In 
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Table 2 Characteristics of respondents 





Europe 

n (%) 


Africa 

n (%) 


Background of respondents 


n = 291 


n = 88 


Clinician (doctor, nurse, social worker, 
therapist, etc.) 


211 (72.5%) 


55 (62.5%) 


Researcher 


r fa c\c\/ \ 

26 (8.9%) 


1 r\ / 1 1 a n/ \ 

10(11 .4%) 


Both 


54 (18.6%) 


23 (26.1%) 


Experience in palliative care 


n = 291 


n = 87 


< 5 years 


1 A1 /*") A ~1C\/ \ 

101 (34.7%) 


32 (36.8%) 


6-10 years 


81 (27.8%) 


o 1 /o /i 1 n/ \ 

21 (24.1%) 


> 1 0 years 


108 (37.1 ) 


o o /o ~7 on/ \ 

33 (37.9%) 


Background of clinicians 


n = 266 


n = 78 


Medicine 


160 (60.2%) 


11 /O O "70/ \ 

31 (39.7%) 


M i i rci run 
IN U 1 bl i iy 


68 D 1 ^ 60/M 


39 (41 n0/M 
jZ ^ I .u /oj 


Other profession 


38 (14.3%) 


15 (19.2%) 


Background of researchers 


n = 80 


n = 33 


Medicine 


46 (57.5%) 


8 (24.2%) 


Nursing 


10 (12.5%) 


8 (24.2%) 


Psychology 


8 (10.0%) 


3 (9.1%) 


Social Science 


5 (6.3%) 


8 (24.2%) 


Other profession 


11 (13.8%) 


6 (18.2%) 



Africa, a third of respondents originated from Uganda 
and about 20% came from Kenya. Further responses 
came from 20 other African countries. About 6% parti- 
cipants replied from countries outside of Europe or 
Africa (e.g. Australia, Canada, Thailand, US, Brazil). 
Their data has been merged into the European and Afri- 
can data set as described in the methods section. 
General use of PROMs in palliative care 
The majority of respondents in Europe (68.1%) and in 
Africa (73.6%) had experiences with PROMs in palliative 
care and more than half of respondents in both conti- 
nents were using PROMs at the time of answering the 
survey (see Figure 1). 



c \ 




Yes, I am Yes, but No, never No, considered 

currently using stopped using considered but never used 
them them 

■ Europe ■ Africa 



Figure 1 Use of outcome measures in palliative care. 

V / 



Main reasons for not using PROMs differed slightly 
between Europe and Africa (see Table 3). In Europe, 
time constraints and patient factors were the main rea- 
sons for not using PROMs (each about 25%) but lack of 
training (20%) and guidance (17%) were mentioned fre- 
quently as well. These reasons were mentioned even 
more often by African respondents whereas patient 
related factors such as tools are too burdensome or do 
not reflect patients' situation played a minor role in the 
views of African professionals. 

Asked under which circumstances respondents would 
start using PROMs, the provision of information, gui- 
dance and training were named most often in both con- 
tinents (Table 3). Only a minority in Europe and 
nobody in Africa thought that they would never use 
PROMs in palliative care. Overall, in Europe time, bur- 
den, and the feeling that measures don't reflect patients' 
situation played a bigger role compared to African 
respondents who mainly described the lack of available 
measures and translations. 
Use of PROMs in clinical care and audit 
A variety of purposes for the use of PROMs in clinical 
care/audit were named by respondents. Main reasons 
for the majority in Europe and in Africa were assess- 
ment of the patients' situation, monitoring changes in 
patients' health status and evaluating the effect of an 
intervention. The latter, assessing families' needs and 
use in audit played a bigger role in Africa compared to 
Europe (see Table 4). 

Advantages of using PROMs were better understand- 
ing of patients' and families' needs, improvement of 
quality of care and help in decision making. 

76% of results of outcome measurement were dis- 
cussed in clinical meetings, 74% were documented in 
patients' records and 73% were used to inform or deter- 
mine treatment or care. Less than half of the respon- 
dents used results for discussions with patients or 
families. 

The overall experience of using PROMs in clinical 
care and audit was rated as good or very good by 70% 
of respondents in Europe and 76.4% in Africa. Only 
around 7% in both continents reported bad or very bad 
experiences. 

Use of PROMs in palliative care research 

The majority of those using PROMs in research in Eur- 
ope and in Africa used it to measure patients' symptoms 
or quality of life and to evaluate the effect of an inter- 
vention (Table 5). Measurement of quality of care 
seemed to be more important in Africa than in Europe. 

Validation of a tool in palliative care population was 
the main factor that influenced researchers' choice of an 
outcome measure in both continents. Access to a tool 
and time needed for completion were additional impor- 
tant factors in Africa. In Europe, comparability with the 
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Table 3 Reasons for not using outcome measures and circumstances starting using them (multiple answers possible) 





Europe n (%) 


Africa n (%) 


Main reasons for not using outcome measures 


n = 201 


n = 63 


Time constraints 


52 (25.9%) 


11 /i 7 rn/\ 
ll (I /.5%) 


Tools are too burdensome for patients & families 


a~j /n /in/ \ 

47 (23.4%) 


~> i~> nn/ \ 

2 (3.2%) 


Lack of training about how to use tools 


41 (20.4%) 


13 (20.6%) 


Tools do not reflect the patient's situation 


t r\ 1 1 c\ /in/ \ 

39 (19.4%) 


2 (3.2%) 


Tools are too burdensome for staff 


~> r 1 1 i nn/ \ 

36 (17.9%) 


~> I a on/ \ 

3 (4.8%) 


Lack of guidance about how to use tools 


~> C / 1 1 AD/ \ 

35 (17.4%) 


1 r /"~> ~> on/ \ 

15 (23.8%) 


Don't know where to get information about tools 


3 I ( I 5.4%) 


o 1 1 i in/ \ 
O (1 2./%) 


Lack of training about how to analyse data from tools 


n r\ / i a nn/ \ 

30(1 4.9%) 


1 ~> ^1 n r\n/ \ 
I 2 (1 9.0%) 


Not enough staff 


nn / 1 a nn/ \ 

30 (14.9%) 


7 (1 1 .1%) 


Limited access (e.g. registration needed) 


1 ~> I r cn/ \ 

13 (6.5%) 


2 (3.2%) 


Language restriction (e.g. tools not translated) 


1 ~\ (r r\n/ \ 

12 (6.0%) 


6 (9.5%) 


Cost constraints (e.g. fees for tools) 


7 (3.5%) 


4 (6.3%) 


Lack of validated version for specific setting (e.g. Africa) 


a I") r\n/ \ 

4 (2.0%) 


in / 1 r r\n/ \ 

10 (15.9%) 


v-ircurnbidnceb 10 bidn use ot outcome measures 


n — 1 Q"3 

n — i ?z 


n — jz 


If more information and guidance were provided about tools 


86 (44.8%) 


26 (50.0%) 


If more training was provided 


58 (30.2%) 


20 (38.5%) 


If I had more time 


56 (29.2%) 


10 (19.2%) 


If appropriate tools were available (e.g. translations) 


42 (21.9%) 


22 (42.3%) 


If I had more staff 


42 (21.9%) 


7 (13.5%) 


I don't think I would ever use tools 


16 (8.3%) 


0 (.0%) 


If I had more money 


9 (4.7%) 


6 (11.5%) 


Table 4 Purpose and advantages using outcome measures in clinical care/audit (multiple answers possible) 




Europe n (%) 


Africa n (%) 


Purpose of using outcome measures in clinical care/audit 


n = 254 


n = 86 


To assess patients' symptoms/needs/problems 


234 (92.1%) 


77 (89.5%) 


To monitor changes in patients' health status or quality of life 


180 (70.9%) 


58 (67.4%) 


To evaluate the effect of an intervention/care/service 


1 72 (67.7%) 


68 (79.1%) 


To facilitate communication in the team 


117 (46.1%) 


35 (40.7%) 


To document patients' characteristics 


116 (45.7%) 


37 (43.0%) 


To assess families' needs/problems 


111 (43.7%) 


51 (59.3%) 


To facilitate communication with patients/families 


85 (33.5%) 


40 (46.5%) 


To assess the care given against standards (in audit) 


59 (23.2%) 


45 (52.3%) 


Other, please state 


9 (3.5%) 


12 (14.0%) 


Advantages of using outcome measures 


n = 233 


n = 78 


To better understand patients' and families' needs (e.g. assessment, screening) 


186 (79.8%) 


72 (92.3%) 


To improve the quality of care for patients and families 


1 72 (73.8%) 


72 (92.3%) 


To make decisions regarding treatment/care 


1 70 (73.0%) 


73 (93.6%) 


To communicate in the team 


147 (63.1%) 


56 (71.8%) 


To evaluate services 


121 (51.9%) 


64 (82.1%) 


To communicate with the patients and families 


102 (43.8%) 


58 (74.4%) 


Other, please state 


16 (6.9%) 


14 (17.9%) 
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Table 5 Purposes for using outcome measures and factors influencing the choice of an outcome measure in research 
(multiple answers possible) 





Europe n (%) 


Africa n (%) 


Purpose of using outcome measures in research 


n = 1 19 


n = 48 


To measure/describe patients' symptoms 


90 (75.6%) 


35 (72.9%) 


To measure/describe patients' quality of life 


77 (64.7%) 


40 (83.3%) 


To evaluate the effect of an intervention/care/service 


77 (64.7%) 


36 (79.2%) 


To assess patients' functional status 


71 (59.7%) 


29 (60.4%) 


To monitor changes in patients health status or quality of life 


57 (47.90/0) 


32 (66.7%) 


To measure/describe patients' quality of care 


44 (37.0%) 


33 (68.8%) 


To screen whether patients meet inclusion criteria 


37 (31.1%) 


15 (31.3%) 


Factors influencing choice of outcome measures in research 


n = 1 18 


n = 46 


Validated in palliative care 


80 (67.8%) 


34 (73.9%) 


Comparability with national and/or international literature 


76 (64.4%) 


20 (43.5%) 


Tool previously used in similar setting/patient group 


70 (59.3%) 


24(52.2%) 


Time needed for completion 


64 (54.2%) 


26 (56.5%) 


Existing translation for my language/country 


63 (53.4%) 


18 (39.1%) 


Access to tool 


59 (50.0%) 


30 (65.2%) 


Validated in patient group (e.g. disease) 


59 (50.0%) 


20 (43.5%) 


Own previous experience with a tool 


53 (44.9%) 


15 (32.6%) 


Cost (e.g. fees to use tool) 


23 (19.5%) 


16 (34.8%) 



national and international literature and previous use of 
the tool in a similar setting or patient group were 
further factors that influence the choice of a tool (see 
Table 5). 

Overall experiences using PROMs in research were 
judged as good or very good by almost two thirds of 
researchers in Europe (64.7%) and more than four in 
five respondents in Africa (82.5%). About a third in Eur- 
ope and 15% in Africa judged their experiences as 
neither good nor bad. Only a minority of 2% reported 
bad experiences. 

Problems and barriers using PROMs in palliative care 

In the open questions, participants expressed a variety 
of problems and barriers for the use of PROMs in clin- 
ical care, audit and research (see Table 6). They can be 
related to patients, staff and the PROMs itself. Patients 
in palliative care are described as a vulnerable group, 
often too ill to fill in PROMs due to the disease or 
cognitive impairment, or find it difficult to understand 
what is wanted of them. Staff barriers relate to gate 
keeping, lack of time, lack of training or general reluc- 
tance to use PROMs in palliative care. Issues that were 
mentioned related to PROMs were access to and 
choice of tools, questions around validity and reliabil- 
ity, adaptation in different languages and cultures, and 
complexity of the measures. In addition to the 
described areas, researchers reported specifically pro- 
blems around data analyses and dealing with missing 
data. 



Future development of PROMs in palliative care 

Respondents also had the opportunity to comment on 
further development of PROMs in palliative care (see 
see Table 6). It emerged that people would prefer simple 
tools relevant to clinical practice. During the develop- 
ment of such tools the service-user perspective needs to 
be taken more into account. Respondents favoured tools 
in an electronic format with the possibilities to choose 
from different modules according to patient situation 
and condition. Many respondents commented that there 
were an ample number of OMs and that our focus 
should be on the improvement of existing measures 
rather than the development of new tools. 
Development of web-based resources 
Answers on development of web resources for the use of 
PROMs highlighted the need to provide interactive con- 
tent with clinical cases and training examples in addition 
to detailed information about specific PROMs and related 
references (see see Table 6). The need for guidance on 
how to use an outcome measure and analyse data was 
expressed. Professionals wish to communicate with each 
other using comments, fora, and sharing content (i.e. 
advice, syntax for analysis). This need for communication 
and sharing information is not met by current websites 
focussing on outcome measurement in palliative care. 

Discussion 

This is the first international survey conducted in Eur- 
ope and in Africa seeking the views of professionals on 
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Table 6 Views of respondents on the use of PROMs in palliative care 

Positive experiences using tools 

• "acceptance of the experience by staff in spite of initial reluctance" 

• "good information to share with staff as affirmation of care and in identifying areas of improvement" 

• "the tools turned out to be exceptionally helpful referring to our negotiations with commissioners" 
Problems and barriers using tools 

• "staff sometimes take the audits as criticism and not as constructive criticism" 

• "some health professionals are not sure how to use them (PROMs) or understand the benefit of them and so consistency can be a problem - 
particularly if trying to look at change over time and if a different person is asking on different occasions. Sometimes the health professionals 
complete the tool but do not look at what it is telling them and use this to help the patient - they are just ticking the box to say that they 
have used them" 

• "scores don't always correspond to how the patient is feeling" 

• "some patient groups have more difficulty than others completing them e.g. hospital inpatients achieved 50% response rate whenever used 
patient completed tool" 

Development of web-based resources 

• "updated info in regard to symptoms management, spiritual care, psychosocial interventions, including case-studies when available" 

• "recommended reading resources (research etc.)" 

• "training opportunities for multi-disciplinary staff members" 

• "list of centres using the tool for exchange of experience (or even creating control groups in terms of research)" 

• "comparative information on different tools to ease choice (as far as possibly in lay-friendly terms); access when possible to PDF or word 
document of the tool; contact details and requirements of use (e.g. payment); link to validation paper and other publications; accounts of 
researchers who used the tool" 

Future development of PROMs in palliative care 

• "don't invent another one! Use/develop the ones we currently have" 

• "I think there is enough already and we should focus on identifying the best and then working out how to implement/integrate them in 
routine care" 

• "if there can be training on how to use the tools it would be better because it is important" 

• "keep them easy, otherwise implementation will be difficult. Tools are only as good as the people who use them, and I often prefer reading 
through someone's narrative account of a situation rather than looking at numbers" 



PROMs in palliative care, their experiences with PROMs 
in clinical care, audit and research, and their perceived 
need regarding future development of measures and 
necessary resources. 

About a third of respondents in Europe and about a 
fifth of those in Africa reported not using PROMs in 
palliative care for several reasons. Only a few respon- 
dents precluded that they would ever use PROMs. This 
suggests a general openness for PROMs even of those 
respondents who don't currently use them. However, it 
might be argued that only people with a positive attitude 
towards PROMs participated in this survey. 

PROMs help professionals in a variety of ways, includ- 
ing better understanding patients' and families' situa- 
tions, monitoring change and evaluating interventions. 
Choice of PROMs is hindered by the huge numbers of 
tools available with many of them not being developed 
for the clinical but rather for the research setting [20]. 
In research, a variety of factors influence choice of an 
outcome measure but validation of the PROM in pallia- 
tive care was most important for European and African 
researchers. Comparability with national and/or interna- 
tional literature plays a bigger role in Europe whereas 
access to tools is an important issue in Africa. 



Only few studies assessed professionals' views on out- 
come measurement [10,11,21]. Young and Maher identi- 
fied resource issues, missing data, communication skills 
and site of questionnaire completion as main problems 
of collecting quality of life data in EORTC clinical trials 
[10]. Oncologists appeared to be willing to use quality 
of life measures but expressed a more positive attitude 
and more willingness to use quality of life data than 
what their reported current behaviour indicated that 
they are actually doing [11]. 

The lack of training and guidance was one of the main 
reasons mentioned by those not using PROMs but also 
emphasized by those regularly using PROMs. If outcome 
measurement should become routine in clinical pallia- 
tive care as has been proposed [8], the development of 
guidance how to use PROMs and of resources for learn- 
ing and teaching seems to be crucial to support profes- 
sionals in this area. Guidance should also provide 
support to overcome barriers such as gate keeping, bur- 
den for patients and access to PROMs. Projects such as 
the Palliative Care Outcomes Collaboration in Australia 
which is a national approach towards the routine assess- 
ment of palliative care using standardised assessment 
tools, is already providing training for professionals [22]. 
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Thus, development of guidance and educational material 
will be an important step to improve professionals' 
knowledge on outcome measurement and is part of the 
work of the PRISMA collaborative [12]. Increased 
knowledge is important but has been challenged as not 
sufficient enough to change behaviour [23]. Thus, such 
materials have to be embedded in multiple strategies to 
overcome different barriers [24]. Another strategy could 
be to develop competency-based training programs 
which are learner- or participant-centered taking into 
account the individual situation and the clinical environ- 
ment of the professional [25]. 

Institutional barriers such as time constraints, and 
gate keeping were more prominent in Europe than in 
Africa. Both factors are not necessarily palliative care 
specific although gate keeping is known to be an impor- 
tant factor for professionals not recruiting patients to 
palliative care research [9]. This survey shows in addi- 
tion that it is also a barrier for the regular use of 
PROMs in palliative care. Time constraints could be 
addressed by training on outcome measurement as it 
might help to reduce time that it takes to complete 
measures. 

To aid further development of outcome measurement 
in palliative care, international and national palliative 
care and professional organisations should be mobilized 
and encouraged to collaborate similar to successful col- 
laborations in oncology, such as the EORTC, the ECOG, 
or the interRAI collaborative in geriatrics [26]. 

Strengths and limitations 

This survey included professionals from two major 
regions, Europe and Africa. Palliative care has developed 
over the last decade in Africa in a more structured way 
[15], and our survey enabled access to the workforce on 
this continent. The web-based design allowed us to 
reach more professionals than would have been feasible 
with a postal survey, both in Europe and in Africa. This 
survey also represents a wide range of professionals 
from a variety of medical, nursing and other back- 
grounds reflecting the multiprofessional and interdisci- 
plinary work in palliative care. This reflects daily 
practice as PROMs are used in a variety of ways and by 
various professionals. Particularly the views of clinicians 
are not often heard in outcome measurement. 

There are some limitations to this online survey. First, 
the results of this survey may be biased as only profes- 
sionals with a positive attitude towards PROMs may 
have participated in the survey. Overall experiences of 
clinicians and researchers with PROMS were similarly 
positive in Europe and in Africa and only a minority of 
respondents in both groups had negative experiences. 
However, about a third of European respondents and 
about a quarter of African professionals stated that they 



were not using PROMs or have stopped using them and 
gave their views on this. Thus, we were able to collect a 
range of views. 

Second, invitations to participate in the survey were 
only sent out via email and not by post. This intro- 
duces bias towards those professionals having access to 
the internet. We assumed that the majority of pallia- 
tive care professionals have internet access at least at 
work as online communication is now part of everyday 
life and even more so in medicine. However, we 
acknowledge that web access may be more problematic 
in Africa compared to Europe. Therefore, the African 
sample could potentially be biased and not fully repre- 
sentative of those working in palliative care in Africa. 
A. As mentioned above, a postal survey would also 
have had limitations and probably reached less 
participants. 

Third, due to limited resources and limited time, the 
survey was conducted in English only. The language 
restriction may be one reason for the low responses 
from European countries other than the UK where 
about a third of respondents came from. Similarly, in 
Africa French speaking countries are potentially under- 
represented in this survey due to the language restric- 
tion. Nevertheless, the survey represents views from 
professionals in southern, central and northern parts of 
Europe, and almost 20 African countries. 

Fourth, participation and completion rate (42 and 
59%, respectively) could have been higher. We sampled 
professionals via national bodies to participate in the 
survey without knowing their actual involvement in pal- 
liative care. Non-responders might have been an 
unknown proportion of professionals without direct 
experience of palliative care provision or opposing out- 
come measurement in palliative care for reasons such as 
patient or staff burden or lack of resources. We cap- 
tured some of these views in our survey but they might 
be under-represented. In addition, web-based surveys 
have known disadvantages with regard to sampling, 
length of questionnaires and access [27]. We tried to 
increase the response to the survey sending two remin- 
ders and offered a low-cost prize draw with book 
tokens. We also paid special attention to the user 
friendliness of the online questionnaire and piloted it 
with a group of professionals. 

It is also important to consider that through sending 
out invitation e-mails with a link to the survey, no infor- 
mation on individual response status could be collected 
[28]. And although we tried to contact as many different 
palliative care associations as possible the distribution of 
different professions is not as equal as hoped, with a 
dominance of physicians and nurses, in addition 
response rate within various regions varied. As the num- 
ber of online surveys may increase in the future, this is 
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an area of relevance for profession-focussed palliative 
care organisations. 

Conclusions 

Our study answered a call for harmonising research and 
best practice across Europe, and in doing so we have 
responded to an urgent need for improved measurement 
in eol care. Our findings indicate that these respondents, 
while mostly committed to measuring outcomes, clearly 
indicated a cry for help in relation to guidance and 
training in outcome measurement in palliative care. It is 
concluded that a coordinated response is now required 
to develop outcome measurement in eol care. Our 
respondents have provided directions in terms of how 
this needs to be achieved, including a need for training, 
guidance, clinically relevant tool refinement. This could 
comprise mobilising national and international palliative 
care and professional organisations to optimise the out- 
comes of the delivery of eol care ensuring that this ser- 
vice is both delivered and measured. 

Additional material 



Additional file 1: Survey invitation letter. Invitation letter sent via e 
mail to members of palliative care associations and international contacts 
with information about the PRISMA project and the online survey, and a 
direct link to the survey. 

Additional file 2: Survey participant information. First screen of 
online survey with more detailed information about the online survey. 



Acknowledgements 

PRISMA is funded by the European Commission's Seventh Framework 
Programme (contract number: Health-F2-2008-201655) with the overall aim 
to co-ordinate high-quality international research into end-of-life cancer care. 
PRISMA aims to provide evidence and guidance on best practice to ensure 
that research can measure and improve outcomes for patients and families. 
PRISMA activities aim to reflect the preferences and cultural diversities of 
citizens, the clinical priorities of clinicians, and appropriately measure 
multidimensional outcomes across settings where end-of-life care is 
delivered. Principal Investigator: Richard Harding. Scientific Director: Irene J 
Higginson. Work package 4 lead: Claudia Bausewein. PRISMA Members: 
Gwenda Albers, Barbara Antunes, Emma Bennett, Ana Barros Pinto, Claudia 
Bausewein, Dorothee Bechinger-English, Hamid Benalia, Lucy Bradley, Lucas 
Ceulemans, Barbara A Daveson, Luc Deliens, Noel Derycke, Martine de 
Vlieger, Let Dillen, Julia Downing, Michael Echteld, Natalie Evans, Dagny 
Faksvag Haugen, Nancy Gikaara, Barbara Gomes, Marjolein Gysels, Sue Hall, 
Richard Harding, Irene J Higginson, Stein Kaasa, Jonathan Koffman, Pedro 
Lopes Ferreira, Johan Menten, Natalia Monteiro Calanzani, Fliss Murtagh, 
Bregje Onwuteaka-Philipsen, Roeline Pasman, Francesca Pettenati, Robert 
Pool, Tony Powell, Miel Ribbe, Katrin Sigurdardottir, Steffen Simon, Franco 
Toscani, Bart van den Eynden, Jenny van der Steen, Paul Vanden Berghe, 
Trudie van lersel. 

Author details 

1 King's College London, Cicely Saunders Institute, Department of Palliative 
Care, Policy and Rehabilitation, London, UK. 2 Deutsche Gesellschaft fur 
Palliativmedizin, Berlin, Germany, formerly African Palliative Care 
Association, Kampala, Uganda. 4 African Palliative Care Association, Kampala, 
Uganda. 



Authors' contributions 

CB, STS, RH and IJH conceived and designed the study; CB, STS, JD, FMP, BD 
and IJH developed the online survey; HB, CB, BD and STS analysed the data; 
CB drafted the manuscript; STS, BD, IJH, RH critically revised the manuscript. 
All authors read and approved the final document. 

Competing interests 

The authors declare that they have no competing interests. 

Received: 5 November 2010 Accepted: 20 April 2011 
Published: 20 April 201 1 

References 

1. Department of Health: Guidance on the routine collection of Patient 
Reported Outcome Measures (PROMs). London: Department of Health; 
2008. 

2. Ferrell B, Connor SR, Cordes A, Dahlin CM, Fine PG, Hutton N, Leenay M, 
Lentz J, Person JL, Meier DE, Zuroski K, National Consensus Project for 
Quality Palliative Care Task Force, Members: The national agenda for 
quality palliative care: the National Consensus Project and the National 
Quality Forum. [Review] [5 refs]. Journal of Pain & Symptom Management 
2007, 33(6):737-44. 

3. Secretary of State for Health: Equity and excellence: Liberating the NHS. 
London: Department of Health; 2010. 

4. Teno JM, Byock I, Field MJ: Research agenda for developing measures to 
examine quality of care and quality of life of patients diagnosed with 
life-limiting illness. Journal of Pain and Symptom Management 1999, 17:81. 

5. Tilden VP, Tolle S, Drach L, Hickman S: Measurement of quality of care 
and quality of life at the end of life. [References]. The Gerontologist 2002, 
42:80. 

6. Noel PH, Frueh BC, Larme AC, Pugh JA: Collaborative care needs and 
preferences of primary care patients with multimorbidity. Health Expect 
2005, 8:54-63. 

7. Sigurdardottir KR, Haugen DF, van der Rijt CC, Sjogren P, Harding R, 
Higginson IJ, Kaasa S: Clinical priorities, barriers and solutions in end-of- 
life cancer care research across Europe. Report from a workshop. 
EurJCancer 2010, 46:1815-1822. 

8. Davies E, Higginson IJ: The Solid Facts Palliative Care. Copenhagen, 
Denmark: WHO Regional Office for Europe; 2004. 

9. Bossola M, Murri R, Onder G, Turriziani A, Fantoni M, Padua L: Physicians' 
knowledge of health-related quality of life and perception of its 
importance in daily clinical practice. Health Qual Life Outcomes 2010, 8:43. 

10. Young T, Maher J: Collecting quality of life data in EORTC clinical trials— 
what happens in practice? Psychooncology 1 999, 8:260-263. 

11. Bezjak A, Ng P, Skeel R, Depetrillo AD, Comis R, Taylor KM: Oncologists' use 
of quality of life information: results of a survey of Eastern Cooperative 
Oncology Group physicians. Qual Life Res 2001, 10:1-13. 

12. Harding R, Higginson IJ: PRISMA: a pan-European co-ordinating action to 
advance the science in end-of-life cancer care. EurJCancer 2010, 
46:1493-1501. 

13. Higginson IJ, McCarthy M: Validity of the support team assessment 
schedule: do staffs' ratings reflect those made by patients or their 
families? Palliat Med 1993, 7:219-228. 

14. Hearn J, Higginson IJ: Development and validation of a core outcome 
measure for palliative care: the Palliative Care Outcome Scale. Quality in 
Health Care 1999, 8:219-227. 

15. Harding R, Higginson IJ: Palliative care in sub-Saharan Africa. Lancet 2005, 
365:1971-1977. 

16. Eysenbach G: Improving the quality of Web surveys: the Checklist for 
Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res 
2004, 6:e34. 

17. Bausewein C, Le Grice C, Simon ST, Higginson IJ: The use of two common 
palliative outcome measures in clinical care and research: a systematic 
review about POS and STAS. Palliat Med 201 1. 

18. Sigurdardottir KR, Haugen DF, Bausewein C, Higginson IJ, Harding R, 
Rosland JH, Kaasa S: A pan-European survey of research in end-of-life 
cancer care. Support Care Cancer 2010. 

19. Krippendorff K: Content Analysis: An Introduction to Its Methodology. 2 
edition. Thousand Oaks: Sage Publications; 2004. 



Bausewein et al. Health and Quality of Life Outcomes 201 1, 9:27 
http://www.hqlo.eom/content/9/1/27 



Page 11 of 1 1 



20. Higginson IJ, Carr AJ: Measuring quality of life: Using quality of life 
measures in the clinical setting. BMJ 2001, 322:1297-1300. 

21. Fung CH, Hays RD: Prospects and challenges in using patient-reported 
outcomes in clinical practice. Qual Life Res 2008, 17:1297-1302. 

22. Currow DC, Eagar K, Aoun S, Fildes D, Yates P, Kristjanson LJ: Is it feasible 
and desirable to collect voluntarily quality and outcome data nationally 
in palliative oncology care? J Clin Oncol 2008, 26:3853-3859. 

23. Prochaska JO, DiClemente CC, Norcross JC: In search of how people 
change. Applications to addictive behaviors. Am Psychol 1992, 
47:1102-1114. 

24. Grol R, Grimshaw J: From best evidence to best practice: effective 
implementation of change in patients' care. Lancet 2003, 362:1225-1230. 

25. ten Cate O, Snell L, Carraccio C: Medical competence: the interplay 
between individual ability and the health care environment. Med Teach 
2010, 32:669-675. 

26. Steel K, Jonsson PV, Dupasquier JN, Gilgen R, Hirdes J, Schroll M, 
Ljunggren G, Carpenter I, Bjornson J: Systems of care for frail older 
persons. InterRAI. Trans Am Clin Climatol Assoc 1999, 110:30-35, discussion 
35-37.. 

27. Wright KB: Researching Internet-Based Populations: Advantages and 
Disadvantages of Online Survey Research, Online Questionnaire 
Authoring Software Packages, and Web Survey Services. Journal of 
Computer-Mediated Communication Wiley; 2005. 

28. Dobrow MJ, Orchard MC, Golden B, Holowaty E, Paszat L, Brown AD, 
Sullivan T: Response audit of an Internet survey of health care providers 
and administrators: implications for determination of response rates. J 
Med Internet Res 2008, 10:e30. 



doi:1 0.1 1 86/1 477-7525-9-27 

Cite this article as: Bausewein et al.: Implementing patient reported 
outcome measures (PROMs) in palliative care - users' cry for help. 

Health and Quality of Life Outcomes 201 1 9:27. 



Submit your next manuscript to BioMed Central 
and take full advantage of: 

• Convenient online submission 

• Thorough peer review 

• No space constraints or color figure charges 

• Immediate publication on acceptance 

• Inclusion in PubMed, CAS, Scopus and Google Scholar 

• Research which is freely available for redistribution 

Submit your manuscript at (^\ RioMM i r pnt ^\ 

www.biomedcentral.com/submit Blomea ^ enirai 



